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Attention: PCT-DO/US 



Re: International Application PCT/DE99/01261 filed April 29, 1999 

Priority Claimed: German AppL No. 198 36 767.8 filed August 13. 1998 



Inventor(s): Walter ROSENBAUM and Frank PFEIFFER 

Title: METHOD AND DEVICE FOR PROCESSING MAIL TO BE 
RETURNED TO SENDER 



Sir: 



Submitted herewith, as the first submission, are the following for the purposes of entering 
the national stage for the USA under 35 U.S.C. 371(c), immediate national examination under 35 
U.S.C. 371(f) being requested. 

X German-language International Application No. PCT/DE99/01261 published as WO 
00/10113. 

X German language International Preliminary Examination Report. 
X Translation of International Application. 
X Formal Drawings (two sheets, Figs. 1-2). 
X Preliminary Amendment. 

X Information Disclosure Statement, International Search Report, German Office Action, 
Form PTO 1449 and references cited. 
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charge the fee to our Account No. 22-0261 and notify us accordingly. 

Please use the following address for corresponding with all counsel of record: 

VENABLE 
P.O. Box 34385 
Washington, D.C. 20043-9998 

Respectfully submitted, 

Robert Kinberg ^ 
Registration No. 26,924" / 
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PCT/DE99/01261 

April 29, 1999 
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Attorney Docket : 32221-168681 RK 



PRELIMINARY 
AMENDMENT 



February 13, 2001 



Assistant Commissioner for Patents 
Washington, D.C. 20231 



Sir: 



Prior to calculation of the fees, please amend the claims in the specification as follows: 
In claim 3, line 1, delete "or 2". 

In claim 4, line 1, change "one of the claims 1 to 3" to -claim 1-. 
In claim 5, line 1, change "claims 1 to 4" to -claim 1--. 



REMARKS 

This Preliminary Amendment is made to eliminate multiple claim dependency. 
Examination on the merits of the application is requested. 

ResfJfctfuUy submitted, 




Robert Kinberg 
Registration No. 26,924 
VENABLE 
Post Office Box 34385 
Washington, DC 20043-9998 
Telephone: (202) 962-4800 
Telefax: (202)962-8300 
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Method and Device fo r Pm crrcinS Mfl ' 1 Ttprng to hp TtetmnrA tn SpnHer 



The invention relates to a method and a device for processing mail items to be 
returned to sender, in accordance with the preamble to the independent claims. 

10 Returning mail items to the sender is necessary if the mail item cannot be 

delivered to the recipient and cannot be forwarded, for example because the new address 
is not known or the sender has provided instructions that the mail item be returned to 
sender if undeliverable. 

The detection of a mail item that must be returned to sender is known from prior 

15 art. If the recipient is not known at the address provided and a new address is not 
available, the mail item will be provided by the delivery person with an automatically 
recognized return stamp. If the mail item contains an instruction, "do not forward, return 
to sender," the return stamp is affixed immediately and without first checking a 
forwarding data file. If a return instruction is recognized during the automatic processing 

20 by the OCR reader and if the process of learning the recipient address, including the 
address interpretation, shows that the recipient is not known under the address provided, 
then this mail item is placed in the category of mail items to be returned to sender. The 
same applies if a return stamp is recognized. 
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The number of mail items to be returned to sender constantly increases and this 
25 service by the Post Office helps large-volume customers, for example mail-order houses, 
to update their address lists. 

At first glance, it appears that this problem is easier to solve automatically than 
the forwarding since the return address is legibly affixed to the mail item and a special 
search of a data bank is not necessary, as is the case for the forwarding. Practical 
30 operations have shown, however, that the OCR reader can clearly recognize sender data 
with much less frequency, so that a video-coding becomes necessary. The reasons for 
this are as follows: 

1. The localization of sender data on the mail item is above all based on aesthetic 
aspects, which differ for each sender. 
35 2. Large-volume customers in particular use special logos, specific lettering styles 
(such as bound handwriting), among other things for advertising purposes as 
registered trademarks. Automatic OCR readers can clearly identify only a small 
number of these. 

3. Additional data is frequently provided as well, such as a telephone number or a 
40 fax number, which leads to non-recognition/rejection by the evaluation algorithms 

developed for the address recognition. 

In order to achieve the most far-reaching online processing of mail items to be 
forwarded, including mail items to be returned, German Patent Application 196 44 163 
45 Al describes a method for the prioritized processing of mail items to be forwarded during 
the video-coding. However, this prioritized processing cannot be used to solve the 
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above-described problem for the case of large- volume customers where numerous mail 
items, containing sender addresses that are hard to read by machine, must be returned to 
sender. 

Thus, it is the object of the invention as stated in the independent claims to create 
a method and a device for processing mail items to be returned to sender. Said method 
and device are designed to strongly reduce the video-coding expenditure for the 
numerous mail items from large-volume customers that must be returned to sender, for 
example a mail-order house, thereby making it possible to have online processing. 

Since every large-volume customer uses a uniform design for its sender data on 
the numerous mail items, the invention starts with the idea of automatically comparing 
the sender data on the image level, so as to detect similarities. Starting with a specific 
degree of similarity, these mail items are then assigned to a single sender. Once they are 
assigned, it is only necessary to videocode a single one of these mail items with respect to 
the sender data. Thus, the time-consuming manual expenditure can be reduced strongly 
and the numerous mail items to be returned to large-volume customers can be processed 
online (sorted), even though an automatic OCR reading of the sender data is not possible. 
The most favorable way to realize this is by immediately video-coding the return mail 
items with sender data that cannot be read automatically and which do not have defined 
similarities to the images of sender regions of previously video-coded mail items. The 
results are then stored together, so that they can be compared to the following mail items 
to be returned and mail items with similar images of the sender regions can be assigned to 
the coded sender. 
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Of course, it is also possible to first examine the mail items to be returned on the 
70 image level to detect similarities in the sender data, to sort them according to the 
similarities and subsequently videocode only one mail item from each stack of mail 
items. 

Advantageous embodiments of the invention are shown in the dependent claims. 
The invention is explained in further detail in the following, with the aid of one 
75 exemplary embodiment and the drawings. 
Shown are in: 

Figure 1 A schematic sequence for processing mail items that must be returned to 
sender. 

Figure 2 A flow chart for a mostly online processing of mail items returned to 
80 sender. 

The decision to "return to sender" can be made for different reasons: 

1 . The mail item contains an instruction by the sender in the form of a printed-on 
message: "If not deliverable, return to sender." If it is determined during the 
address interpretation that the recipient is not listed in the respective address data 

85 file under the address provided, the decision is made to return the mail item. 

2. Following detection and coding of the recipient address, the delivery person 
determines that the recipient does not reside at the given address and no 
forwarding instructions are available or the recipient refuses to accept the mail 
item. In those cases, the delivery person affixes a return stamp to the recipient 

90 address region. If the mail item additionally contains sender instructions for 
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returning the mail item, no check is made to determine whether the recipient has 
moved and/or whether a forwarding instruction exists. 

Figure 1 shows as example a mail item 1 from the "Quelle" mail-order house, 
which could not be delivered to the recipient and therefore was stamped with a return 

95 stamp 2 by the delivery person. During the return process, this mail item 1 is included in 
the distribution process in the same way as a new mail item. First of all, an image of the 
mail item surface is recorded, regions of interest are determined and these regions are 
then classified. The region with recipient data ROI 1 and the return stamp 2 in this region 
are then determined for the above case. As a result of the return decision, the region with 

100 sender data ROI 2 is compared on the image level to the stored images of the recipient 
regions ROI 2 of previously processed mail items to be returned, so as to determine the 
similarities. 

Image processing and detection methods that are known per se are used for this. 
Corresponding methods are disclosed in further detail in: 
105 1. P. Suda et al: Logo and Word Matching. Using a General Approach to Signal 
Registration, 4 th International Conference of Document Analysis and Recognition 
(ICDAR), Ulm, 1997. 

2. A. Rosenfeld and A. Katz: Digital Picture Processing, Academic, chapter 8.3, 
New York, 1976. 

110 3. Robert J. Schalkoff: Digital Image Processing and Computer Vision, John Wiley 
& Sons, 1989. 

Particularly advantageous is the combination of Mellin-Fourier Transformation 
during the auto-correlation, the "Mellin-Fourier-Autocorrelation" (see Item 3, pages 281- 
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286), because it concerns a general, systematic approach, which can be adapted and 

115 expanded easily. 

The mail items to be returned to sender, for which the sender region images have agreed- 
upon degrees of similarity based on the Mellin-Fourier- Autocorrelation, are subsequently 
combined into respective mail item stacks 3. At the end of a day, for example, only the 
sender address region of one mail item is video-coded from each mail item stack while 

120 the remaining mail items in this stack are then assigned to the detected and coded sender 
address and are sorted for delivery to this address. 

The sequence of steps shown in Figure 2 is particularly advantageous for 
achieving the highest possible online processing rate. 

The arriving mail items that are loaded into a sorting machine are first of all separated. In 
125 a reading device, the item surfaces with distribution data of the respectively separated, 
new mail items 10 are scanned 20 and the regions of interest ROI 30 are identified. 
These regions 40 are subsequently classified to determine the recipient region, the sender 
region, the stamp region, the region with sender instructions and the return stamp region. 
Subsequently, the decision 50 for returning the mail item is made. If the mail item is to 
130 be returned, image characteristics of the region are compared to the sender data with 
stored image characteristics to detect similarities, for example by means of the Mellin- 
Fourier- Autocorrelation 60. If no similarities exist to the agreed-upon degree 70, or if no 
image characteristics have been stored 100, then the image of the sender region is video- 
coded 90 and the coding result is stored 100 together with the image characteristics. The 
135 respective mail item is then assigned to the video-coding result 80 and thus can be 
distributed to the sender. If, during the characteristic comparison 70, the agreed-upon 
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similarity with the stored image characteristics of a previously video-coded mail item is 
detected, then the new mail item is assigned to the coded sender address without 
additional video-coding. It is particularly advantageous if the regions containing the 
sender data of known large-volume customers are videcoded prior to processing the mail 
items. The results with the characteristics of the images of these regions, used for the 
similarity analysis, are then stored in a return data file, which is constantly updated 
during the course of the processing of mail items to be returned to sender. 
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Patent Claims 

1 . A method of processing mail items to be returned to sender by scanning and 
storing the images of the mail item surfaces provided with the distribution data and 
identifying at least the regions containing the sender address, characterized in that 
similarities are identified for the image regions containing the sender addresses, that of 
the mail items showing a predetermined degree of similarity for the images of the address 
regions, only one mail item is video-coded while the remaining mail items are assigned to 
the identified sender. 

2. A method according to claim 1, characterized in that the similarities of the 
sender address images are determined with the Mellin-Fourier- Autocorrelation. 

3. A method according to claim 1 or 2, characterized in that the regions containing 
the sender data of mail items from large-volume customers are video-coded prior to 
processing the mail items and are stored in a return data file, together with the 
characteristics determined for the associated images and used for the similarity analysis, 
which return data file is updated during the processing of mail items to be returned to 
sender. 

4. A method according to one of the claims 1 to 3, characterized in that each mail 
item to be returned to sender, for which the sender address cannot be read clearly 
automatically, is checked to determine whether the image characteristics used for the 
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similarity analysis of the region containing the sender data exhibit the predetermined 
degree of similarity with the correspondingly stored image characteristics of the region 
containing the sender data of a mail item with previously video-coded sender address and 
that if this degree of similarity exists, the mail item is assigned to the previously video- 
coded sender address and that if the predetermined degree of similarity does not exist, the 
mail item is then video-coded and the characteristics of the image of the region with 
sender data are stored together with the result of the video-coding. 

5. A method according to claims 1 to 4, for which the mail items are provided with 
(ID) flags and the images of the mail items are stored under the associated ID flags, 
characterized in that the characteristics of the sender data region that are used for the 
similarity analysis are stored together with the result of the video-coding under the 
respective ID flags. 

6. A system for processing mail items to be returned to sender, comprising a device 
for generating and storing mail item images, an OCR reader for the automatic evaluation 
of the identification of the regions of interest, such as sender address, recipient address, 
return to sender instructions or return stamp, with the aid of the stored images, and for 
running a result check in a name/address data bank and a return data file, further 
comprising a device for video-coding the images or the regions of interest on mail items 
with result checks, said device having at least one video-coding station, one image 
controller for controlling the data flow between the OCR reader and the device for video- 
coding, which supplies the corresponding mail item images or only the respective sender 
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address regions to the video-coding device if the sender addresses cannot be read clearly, 
and which feeds the video-coding results to the OCR reader so that they can be stored 
together with the associated image, characterized in that the OCR reader is provided 
with a device for determining similarities between images, which device contains images 
of the sender addresses on the identified mail items that are to be returned to sender, that 
when detecting a fixed degree of similarity between stored images characteristics of the 
region with sender data of one of the previously video-coded mail items and the image 
characteristics of a new mail item, the new mail item is assigned to the coding of the 
previously video-coded mail item and, if the predetermined degree of similarity is not 
reached, the respective image of the sender address of the new mail item is supplied via 
the image controller to the device for video-coding and the coding result is transferred to 
the OCR reader for storage together with the image characteristics of the sender address 
image, which are used for the similarity analysis. 
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As a below named inventor, I hereby declare that my residency post office address and citizenship are as stated below next to my 
name, and that I believe I am the original, first and sole inventor (if only one name is listed below) or an original, first and joint inventor (if 
plural names are listed below) of the $ubj*ct matter which i$ claimed and for which a patent is sought on the invention entitled METHOD 

AND DEVICE FOR PROCESSING MAIL TO BE RETURNED TO SENDER — the specification of which 

[ X ] was filed under the Patent Cooperation Treaty an April 23. 1999 ^as International Application No. PCT/DE99/Q 1 26 1 f the United 
States of America being designated. 

I hereby state that I have reviewed and understand the contents of the above-identified specification, including the claims, as 
amended by any amendment referred to above. 

I acknowledge the duty to disclose to the Patent and Trademark Office all information known to me to be material to patentability 
as defined in Title 37, Code of Federal Regulations, §L5G(a), 

I hereby claim foreign priority benefits under Title 35, United States Code, § 1 19 of any foreign applications) for patent, utility 
model t design or inventor's certificate listed below and have also identified below any foreign application^) for patent, utility model, 
design or inventor's certificate having a filing date before that of the application^) on which priority is claimed: 

__ Prior Foreign Application^) , Priority Claimed 

Number Country Date Filed Yes No 

198 36 767.S ^ Germany^ August 13. 199S ^ X 

1| I hereby appoint the following attorneys and agents to prosecute this application and to transact all business in the Patent and 
^irademark Office connected therewith: George H. Spencer (Re$. No, 13,038) . Richard L. Aitken, f Rea. No. lSJ2£ n. Robert J. Frank 
*$UgJfoU2J121 Norman N. Kunitz ( free. No, 20.SMV»_ Gabnr J, Kelemen r Reg^No. 21.0161 Laurence J, Marhoefer, ( Reg, No, 21,091) , 
" Carina V\ Schnelter (RejJ4o^2&2^), Robert Kinberg (RfgJ&J2k224}, Allen Wood ( Reg. No. 28,134) , John P. Shannon, (Reg. No, 
9aj6^]lftQti E. McSnTT TReg. No. 29.565V Ashlev J. Wells f Reg, No. 29,8471 R ichard D. Schmidt (Reg. No. 31 .30 1\ James R* 
jfijurci^^ A, Gollin (Reg. No. 3 1.957). Catherine M, Voorhees ( Re& No. 33,074), Robert S, Babayi (Reg. No. 

jj g»4711 Gary L. Shaffer (RggJ^o, 34.502), r all at Suite 1000. 1 201 New York Avenue. N.W:. W ashington, D .C T 20005-39 l7 t Telephone : 
^ g02)962-430j> . Telefax : (202) 962-83QQ . 

S? * Address ^corresgcmdenceto VENABLE, P.O. Box 34385, Washington, D.C- 20043-9998... 

The undersigned hereby authorizes the U,S. attorneys named herein to accept and follow instructions from the undersigned's 
|||$ignee, if any, and/or, if the undersigned is not a resident of the United States, the undersigned's domestic attorney, patent attorney or 
fgitent agent, as to any action to be taken in the Patent and Trademark Office regarding this application without direct communication 
i&nveen the U.S. attorneys and the undersigned. In the event of a change in the per$on(s) from whom instructions may be taken, the U.S. 
^tomeys named herein will be so notified by the undersigned. 

I hereby declare that all statements made herein of my own knowledge are true and that all statements made on information and 
13 'are believed to be true; and further that these statements were made with the knowledge that willful false statements and the like so 
Irnaue are punishable by tine or imprisonment, or both* under §1001 of Title 18 of the United States Code and that such willful false 
statements may jeopardize the validity of the application or any patent issued thereon* 
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